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Is it possible to find any specificity of research carried out by psychologists in their own country, 
justifying publication also in their native language? The question posed by Brzezinski (2014) 
requires considering different types of psychological research: studies regarding universal issues 
have to be diffused in a shared scientific format and in an international language, while studies 
regarding specific applicative aspects, linked to particular contexts, can be published in a language 
easily accessible to the psychologists living in a specific country. This kind of research and publi- 
cation, if conducted with correct methodology, should not be considered of secondary scientific 
value. As regards the question of what indices should be used to determine a psychologist’s posi- 
tion in world science, I will discuss the assumption of “objectivity” in this kind of evaluation: 
objectivity is very difficult to obtain using numeric criteria, while pre-determined scientific and 
teaching criteria and thresholds could be more suitable to achieve a full evaluation for habilitation 
or for other scientific or academic purposes. As regards reactions to scientific pathologies such as 
fabrication, falsification, or plagiarism of data, meta-analysis based on cumulative research on 
research could permit to detect unexpected outliers and to discover “file drawer effects,” allowing 
replications and enhancing respect for scientific rules and ethics. 
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The answers to the first two questions posed by Brzezinski (2014) require 
considering different types of psychological research. Theoretical and experi- 
mental research regarding processes (cognitive, emotional, neuropsychological, 
interpersonal) that are basically universal has to be realized and diffused accord- 
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ing to methodological guidelines shared all over the world, and also in a format 
and language universally accepted (i.e., APA style, English language), indepen- 
dently of where the author(s) and the publisher are located. 

Experimental and quasi-experimental studies regarding specific applicative 
aspects, more interesting and useful in a particular country, should be conducted 
according the methodological guidelines indicated above, but they may be pub- 
lished in a place and in a language accessible to the psychologists living in that 
specific country (e.g., Polish or Italian). 

I think the differentiation among different types of research — and the publi- 
cation of the results — should take into account not only the context (where the 
study is conducted) or the purpose (scientific purposes or the formation of spe- 
cialists for psychological practice) but mainly the object of scientific work. As 
examples, I would quote applicative studies: 

— in the educational field, referring to very particular systems of formation 
(e.g., modalities to treat learning disabilities or vocational counseling in specific 
school settings); 

— in forensic psychology, since laws and procedures concerning certain is- 
sues (e.g., adoption, medically assisted procreation, separation and divorce, etc.) 
are very peculiar to specific countries; 

— in the psychometric field, e.g. adaptations of psychological tests in national 
samples. 

The results of these studies could be less attractive for international scientific 
publishers (except journals devoted to intercultural studies) and more appealing 
to scholars who have an interest in the particular problem addressed. This re- 
search could be published (surely, without derogating from universally valid 
methodological norms): in national peer-reviewed journals (avoiding “bookbind- 
er’s syntheses” or publishing houses located “in a brother-in-law’s shed”); in 
a language easily accessible to all the psychologists interested in using these 
specific results (some national journals adopt a more expensive but useful double 
language format). 

The same could be said for scientific debates on themes specific for a nation- 
al context: for example, on issues regarding psychologists in schools, job place- 
ment centers, work and organizational problems, prevention programs for health, 
and so on. Also methodological issues could be treated in a national language to 
disseminate updated scientific knowledge among students and practitioners. 

Many years ago, when I published a textbook on meta-analysis resuming the 
issue of guidelines for conducting scientifically cumulative studies and reporting 
the main results in several applicative fields (Di Nuovo, 1996), I chose the Italian 
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language and an Italian publisher. I preferred to publicize a methodology unfami- 
liar in Italy at that time in this way rather than submit my work to a prestigious 
international publishing house, since many good studies on this subject had 
already been published in English. 

It is important not to prejudicially assume that this kind of research and pub- 
lication — if conducted with correct methodology — is of secondary scientific 
value, also for the purposes of academic evaluation; in this way the participation 
of young researchers can be encouraged. 

Therefore, I agree — on the conditions specified above — with Brzezinski’s 
positive answer to the second question: “Should psychologists publish (also) in 
their native language and should they publish their work also in their own coun- 
try?” The scientific publication in the native language, regarding the themes 
outlined above, is useful for preventing all the scientific knowledge of the psy- 
chologists working in a specific country, as well as the updating of the acquired 
information, from being based on the reductive translations of Wikipedia (or 
similar) entries into the language of that country. 

Regarding Question 3, “What indices are used (or should be used) to deter- 
mine a psychologist’s position in world science?”, I have no trust in any kind of 
“number games,” deriving from some bizarre combination of bibliometric indic- 
es. Numbers (e.g., indices based on citations, such as the h-index, the g-index, or 
others) are linked to criteria that are not “objective,” as they are naively pre- 
sumed to be. In fact, citations depend on: 

— the possibility of publishing in open-access journals (often requesting to 
pay, thus adding a financial argument to the scientific one); 

— the participation in big networks of researchers, with more facility of being 
involved in the game (enterprising?) of reciprocal citation; 

working in scientific fields (e.g., neurosciences) where the system of quo- 
tation and the number of (real or fictitious) coauthors allow to obtain more 
citations. 

Citation indices should be weighted by the academic age of the researcher 
and related to specific fields of research, since publication policies are very dif- 
ferent between specific fields of a complex discipline, such as history of psy- 
chology, neuropsychology, methodology, and educational, social, or forensic 
psychology. An example of this difference is that monographs are preferred in 
some of these fields as the outcome of research work, while brief articles with 
data from single experiments are favored in others. How could numerical indices 
compare such different worlds, with dissimilar theoretical approaches, methods, 
academic traditions, and practices? 
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In Italy, a very complex system of “medians” (i.e., median values of indices 
obtained by the whole population of researchers in each scientific field) has been 
devised for habilitation applications, with many debates and diatribes about crite- 
ria fair for some candidates and subgroups and unfair for others. In some scien- 
tific fields bibliometric indices were taken into account in building these “me- 
dians” (i.e., the quantity of articles and the sum of citations in ISI or Scopus 
journals, the h-index), while nonbibliometric criteria (e.g., the number of books 
or articles also in Italian) were used in other fields, such as sociology, philoso- 
phy, or pedagogy. The inclusion of psychological disciplines — general, educa- 
tional, and social psychology, work, dynamic, and clinical psychology, including 
methodologies — among sciences using bibliometric criteria for academic evalua- 
tion provoked markedly biased outcomes inside the heterogeneous world of psy- 
chology, but particularly unfair if compared with the abovementioned neighbor 
disciplines, where more loose criteria were used. 

In general, despite what is generally presumed, “objectivity” is very difficult 
to achieve using numeric criteria. It can be added that positive citations are not 
distinguished from negative ones, and falsifications and improper procedures 
(Brzezinski’s Question 4) are not easy to detect by means of automatic indices 
alone. In order to assure a fair evaluation of psychologists’ position in world 
science, I would prefer a more traditional procedure in which a commission of 
experts (with the aid of external reviewers) would evaluate applicants for habili- 
tation or other scientific or academic purposes, taking into account several crite- 
ria, such as: correctness and updating of the theoretical background of the overall 
research aims stated in the curriculum vitae; coherence of the field of study; ap- 
propriateness of methods and techniques; quality and originality of results; type 
of publications and places of publication; citations related to each specific type 
of publication. 

In the case of applications for academic positions, the capacity for and exper- 
tise in applying scientific results to teaching should also be evaluated. It is not 
numerical indices that are useful (numbers often obscure substantial quality with 
artificial objectivity!) but a thought-minded and critical appraisal, made by 
a competent and responsible commission, of the whole curriculum vitae — i.e., 
the scientific, teaching, and professional history — of each candidate. 

In conclusion, I agree with the statement found in DORA (2012): “do not use 
journal-based metrics, such as Journal Impact Factors, as a surrogate measure of 
the quality of individual research articles,” and with the conclusions of the Foun- 
dation for Polish Science (2014): “peer review evaluation should definitely be 
used, in which carefully selected academics assess the originality of individual 
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scientific contributions of other researchers, while the h-index and the number of 
citations may play an auxiliary role.” 

As regards Question 4, concerning the conclusions from and reactions to 
scientific pathologies (i.e., fabrication, falsification, plagiarism), Brzezinski’s 
opinion is that psychologists should make their raw data available and share 
them with other researchers, if requested, to permit external replications of em- 
pirical studies. I suggest that meta-analysis, based on cumulative research on 
research, could permit to explore the distribution of experimental effects ob- 
tained in specific fields and issues, detecting the outliers and explaining the 
variance among studies on the same topic using methodological variables as 
mediating predictors (Schulze, 2004; Borenstein, Hedges, Higgins, & Rothstein, 
2009). It is possible to discover the “file drawer effects” (Rosenthal, 1979) by 
directly asking authors for information about unpublished research, besides pro- 
viding raw data for implementing cumulative analyses. Such control of the litera- 
ture could be useful also for allowing replications and enhancing respect for 
scientific rules and ethics. 
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